Task: Create Test Specifications (AST)
Purpose
The creation of the test specifications per test unit.
Relationships
RolesPrimary Performer: Additional Performers:
Outputs
    Process Usage
    Main Description

    Method of operation

    The testers specify the necessary tests for the test units in the test plan. After completion, the test specifications are placed under configuration management. Depending on the test type and test technique selected for the test unit, this activity may consist of anything from the creation of a checklist to the design and specification of test cases according to a test design technique or to the design of a test with other techniques. The possibilities are further explained below. Explanations are also given of a scalable regression test and of the relationship between this phase and exploratory testing.

    In the course of this activity, problems may arise with the test basis. Roughly, these can be categorised as follows:

    • Defects - As with the testability review, the testers may find shortcomings and/or ambiguities in the test basis. The testers create a defect report on this. Via the defects procedure, it is passed to the test basis supplier, who can then solve it.
    • Absence of test basis - If the testability review has been insufficiently executed, it may only appear at this stage that certain parts of the test basis are missing or not detailed enough, so that they are not, or not sufficiently, testable. The same types of measures as adopted with the testability review may be considered.
      With iterative or agile system development, the test basis is often not 100% complete at the start of the iteration, but is completed during the iteration. Besides the above mentioned measures, it is advisable to carry out a minimal testability review with each addition to the test basis before specifying tests based on the addition.
    • Unstable test basis - If the supplier of the test basis makes regular changes to it, for example because of defects or change proposals, this makes for an unstable test basis. With every change, the testers have to examine the relevant test specifications to see whether adjustments are necessary. These reworking operations are always difficult to estimate in advance. The test manager is well advised to arrange a certain level of reserve budget and time for this when creating the test plan. If these are exceeded, the project management should be notifi ed that more time and finances are required. Other possible measures are to defer the specifying of the tests for the unstable parts in the plan or to create the logical test cases, but to delay the physical makeup of them until the test basis is (more) stable.

    Test design techniques

    Test design techniques are employed in the creation of the test cases. They enable the testers to derive test cases that achieve a particular coverage from a test basis in a clear, transferable and reproducible manner. A test case consists of a description of the initial situation, the test action and the predicted result. The specifying of test cases according to a test design technique follows the following general steps:

    • 1. Identifying test situations
    • 2. Creating logical test cases
    • 3. Creating physical test cases
    • 4. Establishing the starting point
    • 5. Creating the test script.

    The concepts in these steps are explained in more detail in section Essential Test Design Concepts, including examples. Further explanation of the steps themselves is given in "Steps" section below.

    A choice has to be made in the establishment of the result of these steps, which collectively form the test specification. Generally, two options are possible. The first is to put everything together in one file, usually a Word document or spreadsheet. The second is to make a split in the design steps (identifying test situations and creating logical test cases) and the steps required for test execution (the rest, resulting in the script). The advantage of the first is that maintenance on the test cases is made easier, since everything is together. The second option does not have this advantage, but does have two others. The testing operative (e.g. a user who has to carry out the test but who has not designed it) is not distracted by the design steps and only sees the information that is necessary to carry out the test. The second advantage is that the information required for the execution can be documented in a form other than word processing or spreadsheet, such as a testware management tool or a database or directly in a tool for automated test execution.

    Checklists

    Besides the specifying of test cases, many tests take place with the aid of checklists. These are used with simple functional tests, but also for the static testing of e.g. maintainability, manageability, user-friendliness or security. While a checklist is usually specific to the situation, testers often use a general checklist as a basis and make specific adjustments to this. The general checklists may be supplied from within the organisation (by the test department) or from the literature or via the Internet. Various examples of checklists for testing certain quality characteristics can be found at www.tmap.net. The creation (and execution) of a checklist requires a competent tester with the necessary knowledge of the object part or characteristic under test. It is therefore advisable to have the checklist reviewed.

    Other techniques

    Apart from the specifying of test cases according to test design techniques and the creation of checklists, other techniques are possible that do not fall into either of the above categories. These techniques mainly apply to the testing of quality characteristics, such as portability, usability, performance and security.

    Products

    • Test basis defects
    • Test specifications (checklists, test cases, test scripts)

    Techniques
    Tools

    • Defect management tool
    • Test design tool
    • Model-based testing tool
    • Testware management tool
    • Automated test execution tool
    • Performance, load and stress test tool.
    Steps
    1. Identifying test situations
    Each test design technique is aimed at achieving a certain coverage in order to find particular types of defects. In this, the first step is to distinguish the situations in the test basis that are to be tested. The test basis consists of, for example, the system requirements, the functional design, the user manual and/or the administrative procedures. This test basis is gone through and each situation to be tested is identified in it. The test design technique prescribes, as regards situations to be tested, for example that the true and false situation is tested in respect of every condition, or that each input field is tested with a valid and invalid input value. Since the distinguishing of test situations is different for every test design technique, this step contains the biggest differences between the various test design techniques.

    2. Creating logical test cases
    In principle, a test case goes through the relevant part of the test object (e.g. a function) from beginning to end, covering one or more test situations in doing so.

    The logical test cases clearly demonstrate:

    • What the test situations are
    • That all the defi ned test situations have been covered by test cases.

    The creation of logical test cases may be skipped if the test situations can be directly executed and assessed one by one.
    3. Creating physical test cases

    In practical terms, the physical test case describes what needs to be done, in which the three basic elements of a test case are recognisable: initial situation, action and predicted result. In the physical makeup, each situation to be tested, or logical test case, is completed with as much detail as possible, so that later, during the test execution, the work can be performed as efficiently as possible. Obviously, the choice of physical makeup is based on the preceding steps, but it is also influenced by agreements on, for example, the use of a certain (central) starting point, e.g. a (blind) copy of the production database. It should also be agreed that the various tests should not be able to disrupt each other during the execution, for example because different test cases use the same data (competition). If one test case is “Remove order lines” while the other is “Change order lines”, there is a strong chance that the predicted and justifi able results will not be realised. A solution is, for example, to agree particular series of numbers for the order rules for the sake of the testing of various functions. In this connection, it is also advisable to create no dependencies between test cases. If the execution of one test case is conditional for the execution of another, the failure of the first test case will also mean that the second test case cannot be executed. Only when the defect that is blocking the first test case is solved can the second test case be executed. If this again results in a defect found, it can lead to extra and unnecessary reworking operations. 

    Section “Test situation, test case and test script” in Essential Test Design Concepts contains a number of considerations as to how far the tester should go in the physical makeup of test cases.

    4. Establishing the starting point
    To be able to execute a test case, an initial situation is necessary. Initial situations often contain the same data for several test cases. Such data are therefore included in a so-called starting point for the entire test and not separated for each test case. This starting point is made ready prior to the test execution. A step further is where the starting points for various tests may also show a (big) overlap. For that reason, it is often a case of one or more central starting points applying to several tests. This is described extensively in Define Central Starting Points (AST).
    5. Creating the test script
    As a last step, one or more test scripts are produced. In this, the test actions and checks of the physical test cases are described in a sequence that best suits test execution. In this, the test cases should not be able to disrupt each other. The test script as such is the stepped plan for the test execution and also offers the possibility of progress monitoring. The physical test cases and the starting points naturally form the basis for the creation of the test script. Considerations for combining test cases in a single script are described in section "Test situation, test case and test script" of Essential Test Design Concepts.

    The generic content list of a script is as follows:
    • Unique identity, consisting of:
      • Version
      • Creator
      • Test basis, including version.
    • Preparing the starting point - For example, by setting the system date, restoring a particular backup and adding particular test data
    • Test actions and verification - The physical test cases in a sequence suitable for execution, with the necessary initial situation for each test case, action and result verification. When an appropriate starting point is established, nothing more is usually required to be done in respect of the initial situation.
    • Cleaning up the environment - Ensure that the results of the executed test are cleaned up if necessary so that they do not get in the way of other testers (remember, for example, to restore the system date).

    The above says nothing about how the script is stored, for example in a Word document, spreadsheet, automated test or a database. Section "Test situation, test case and test script" of Essential Test Design Concepts supplies a number of possibilities and considerations.

    In some cases, the support of tools is possible, for example, tools for generating test data for deriving test cases, or even model-based test tools (see Types Of Test Tools).
    More Information
    Checklists
    Tool Mentors

    Copyright © 2006, Sogeti Nederland B.V. All rights reserved.